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INTRODUCTION 

We  consider  dichotomous  responses  by  subjects  to  a  set 
of  test  items,  when  the  responses  are  assumed  to  follow  the  two 
parameter  logistic  model.  Under  the  assumption  that  ability 
parameters  are  randomly  sampled  from  a  normal  distribution  with 
unknown  parameters,  empirical  Bayes  types  estimates  of  ability 
and  item  parameters  were  proposed  and  illustrated  for  the  one- 
parameter  logistic  (Rasch)  model  by  Rigdon  and  Tsutakawa  (1983) . 

Here  we  will  consider  one  of  these  estimators,  namely  MLF,  where 
the  extension  to  multiparameter  item  response  models  is  straight¬ 
forward  and  asymptotic  methods  are  available  from  the  theory  of 
maximum  likelihood  estimation. 

A  standard  procedure  for  estimating  both  ability  and  item 
parameters  in  the  absence  of  prior  distributions  is  maximum 
likelihood  (cf.  Lord  1980).  Maximum  likelihood  estimation  when 
the  ability  parameters  have  a  common  prior  distribution  have 
been  studied  by  a  number  of  authors,  including,  among  others, 

Bock  and  Lieberman  (1970),  Anderson  and  Madsen  (1977),  Sanathanan 
and  Blumenthal  (1978) ,  and  Bock  and  Aitkin  (1981) .  The  latter  two 
papers  use  different  forms  of  the  EM  algorithm  (Dempster,  Laird, 
and  Rubin  1977)  to  greatly  simplify  what  previously  appeared  to  be 
an  insurmountable  numerical  problem. 

This  paper  shows  that  by  using  the  extended  form  of  the  EM 
algorithm,  which  does  not  require  the  presence  of  sufficient  statistics, 
the  computational  difficulty  is  reduced  to  solving  a  series  of  2 
equations  with  2  unknown,  corresponding  to  the  2  parameters  of  the 
logistic  curve.  Although  the  equations  must  be  numerically  solved 
by  seme  iterative  method,  the  task  isa  vast  simplification  over 
methods  where  all  parameters  are  found  simultaneously.  Under  the 
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assumption  that  the  ability  parameters  are  independent  and 
identically  distributed,  the  classical  maximum  likelihood  theory 
applies  and  the  inverse  of  the  observed  information  matrix  may  be 
used  as  an  approximate  covariance  matrix  of  the  estimated  item 
parameters.  The  successful  use  of  our  method  requires  a  fair 
amount  of  computation.  Most  of  the  computational  burden  is  in 
accurately  evaluating  a  large  number  of  single  intergals,  which 
are  basically  expectations  with  respect  to  posterior  distributions 
of  abilities. 

After  describing  our  model  and  assumptions,  we  discuss  the 
implementation  of  the  EM  algorithm  to  our  problem.  The  equations 
at  each  iteration  of  the  algorithm  is  then  related  to  the  likeli¬ 
hood  equations.  Responses  to  a  questionnarie  on  arthritis 
knowledge  will  then  be  used  to  illustrate  our  method  and 
simulations  will  be  used  to  illustrate  the  reliability  of  the 
observed  information  matrix  for  assessing  the  errors  in  the  item 
parameter  estimates.  Expressions  of  derivatives  needed  to  carry 
out  the  computation  are  summarized  in  the  last  section. 


ITEM  RESPONSE  MODEL 

Consider  n  subjects  responding  to  k  test  items  designed  to 
measure  the  ability  of  the  subjects  in  a  particular  area.  Let  Y 
be  an  n  x  k  matrix  of  binary  responses  where  Y„  =  1  or  0  accordingly 
as  the  ith  subject's  response  to  item  j  is  correct  or  incorrect, 
i  *  l,***,n  and  j  =  l,***,k.  A  general  framework  in  which  to  study 
such  item  responses  is  to  consider  some  probability  model, 


(2.1) 


.1  -  ' 


I 


depending  on  a  real  valued  ability  parameter  of  the  ith  subject, 
ei»  and  a  real  or  vector  valued  item  parameter  of  the  jth  item. 

Given  0T  =  (6^,***,9n)  anc*  £T  =  ***'^k^'  w^ere  T  denotes 

transpose,  conditional  independence  among  the  responses  in  Y  is 
assumed,  so  that  the  joint  probability  of  Y  =  y,  given  (9,  £)  is 

n  k  Y.  .  1-y.  . 

=  n  n  p.J3  (l  -  p. .)  1].  (2.2) 

1=1  j=l  13  13 

We  further  assume  that  9  may  be  treated  as  a  random  sample  from 
some  prior  distribution  with  pdf  p(0jx)  having  parameter  X. 

In  this  paper  we  shall  be  discussing  the  estimation  of  £  in 
the  special  case  of  the  two-parameter  logistic  model,  defined  by 


p(yijl9i,aj,Sj 


y .  .  0.(6.  -  6  . ) 

D  1  3 


1  +  e 


(2.3) 


y^j  =  0,1,  where  is  now  the  vector  (0^,6^)  with  0  <  <  °° 

and  -  «  <  g.  <  •.  For  the  prior  of  9,  we  consider  the  normal 
•  3 

.  2  2 
distribution  with  mean  y  and  variance  o  so  that  X  =  (u,a  ). 

However,  due  to  the  nonuniqueness  of  this  parametrization  we  shall 

2 

add  the  constraints  y=0  and  0  =1.  We  will  thus  assume  that  the 
prior  pdf  of  9  is 

p(6)  =  (2tt)'1/2  exp(-62/2),  (2.4) 

-op  <  0  <  oo.  it  can  be  shown  (Tsutakawa  1982)  ,  that  there  is  an 


equivalence  between  this  parametrization  and  the  more  familiar  one 


where  the  constraints  are 


l  6.  =  0 

j=l  3 


and  n  a.  =  1, 

j-1  3 


with  no  restriction  on  (p,a^). 

For  the  above  model,  the  joint  distribution  of 

~  1  9 

T 

-  (Yn'  •  •  •  given  by 


fty^ejo  =  P(e.)  n  P<y..|0.,a  ,8.) 
x  ~  1  1  J  J 


and  the  joint  distribution  of  (Y,0)  by 


f(X'®l£}  *  n  f(yi»e,U>- 

j=l  1 

Thus  the  marginal  probability  of  response  y.  for  the  ith 

<v  1 


p<xll£»  *  I 


and  for  all  subjects 


n 


P(y|5)  =  n  P (yf | 5 ) . 

~  ~  i_i  5,1  1  ^ 


The  loglikelihood  function  of  £T  =  {(a^B  (ak,Bk)} 

given  by 

n  ,  k 

L(c)  =  I  log  p(e  )  n  P(y .  .  |e  .  ,a  .  ,8  .)de, . 

~  i=i  ;  1  -j=i  *0  i  j  j  i 


9  ^ ) ,  where 

(2.5) 

(2.6) 

subject  is 

(2.7) 

(2.8) 

is  thus 

(2.9) 


PARAMETER  ESTIMATION  VIA 


THE  EM  ALGORITHM 

In  this  section  we  outline  the  use  of  EM  algorithm  for 
obtaining  £ /  the  maximum  likelihood  estimate  of  £ ,  when  the 

a* 

loglikelihood  function  is  given  by  (2.9).  Basically  the  algorithm 
consists  of  iteratively  deriving  the  value  of  £  which  maximizes 
the  posterior  expectation 

T  (£ )  *  E{  log  f  (y  r8  |  £)  |  X,£0},  (3.1) 

given  y  and  a  provisional  estimate  £°,  where  £°  is  initially 

A 

some  approximation  to  £  and  is  replaced  at  the  end  of  each 

A* 

iteration  by  the  maximizing  value  of  £ . 

A* 

*  For  the  two-parameter  logistic  model  (2.3),  each  iteration 
reduces  to  solving  the  equations. 


d6  , 

1+  exp{-a j  (9i“6i)  }  l 


(3.2) 


n 

l 

i=l 


i3 


0  .  p  ( 0  .  ! 
i 


Xi 


-£°)de, 


'i-  ill 


9iP(9iiXi'r  > 

1+  exp{  -a  j  (0i-6i)  }  0i' 


for  (0^  ,0 j)  , 


separately  for  j 


1, . . . ,k,  where 


the  posterior  pdf  of  9 . ,  given  C  =  5°  and  observation  y.,  i=l,...,n. 
The  solution  at  the  end  of  each  iteration  is  used  as  the  value 

o 

of  £  in  the  next  iteration,  and  the  process  continues  till  con¬ 
vergence  is  attained. 

Since  the  solution  to  the  equation  (3.2)  has  no  simple 
expression  it  must  be  derived  numerically  by  iterative  methods 
such  as  the  one  by  Marquardt  (1963) .  Moreover,  the  integrals  must 
be  evaluated  by  numerical  techniques  such  as  Gauss-Hermite .  The 
expressions  for  the  derivative  needed  for  the  implementation  these 
methods  are  summarized  in  the  last  section. 

It  is  instructive  to  compare  equations  (3.2)  with  the  like¬ 
lihood  equation  which  £  satisfies.  The  likelihood  equations, 
which  are  obtained  by  setting  the  first  partial  derivatives  of 
(2.9)  with  respect  to  and  B^  equal  to  zero,  are 


n  n  r 

l  yAj  -  l 

i=l  i=i 


p(0  • I y, *5) 

X  X 


+  exp{-a j (0^-B j ) }  d0i  ' 


(3.4) 


n 


ihYii  J 


9ip(0i I yi'£) d0i 


n 

l 

i=l 


0iP(0i I yi f£) 

1  +  exp{ -a j (0i-Bj) >  d9i' 


j=l,...,k,  where  £  ={  (a^»B^)  ,  ~.,  (a^'S^)}  The  only  difference 
between  the  two  systems  of  equations  (3.2)  and  (3.4)  is  in  the 
role  £  and  £°.  In  (3.2)  £°  is  a  fixed  known  value  at  each  iteration 
whereas  in  (3.4)  £  is  an  unknown  to  be  solved.  Solving  (3.4) 
directly  requires  simultaneously  finding  2k  unknowns  in  2k  equations 


’  -  *  •  •  <  •  c 


V 


On  the  other  hand,  each  iteration  of  the  EM  algorithm  only 
requires  finding  a  series  of  2  unknowns  in  2  equations.  Note 
that  once  convergence  is  attained  using  the  EM  algorithm,  the 
successive  values  of  £°  remain  unchanged  and  satisfy  the  likelihood 
equations  (3.4). 

ERRORS  IN  PARAMETER  ESTIMATES 

Since  the  vector  valued  observations  Y^,Y2'.**,?  are 
independent  with  a  common  distribution  (2.7),  we  appe  t:o 
the  asymptotic  properties  of  the  maximum  likelihood  e  ator 
to  assess  the  reliability  of  £.  For  large  samples,  £  is 
approximately  normal  with  mean  £  and  covariance  I-1(£>,  the 
reciprocal  of  the  Fisher  information  matrix  I.(£)  .  In  practice 
£  is  unknown  and  I(£)  is  difficult  to  compute.  We  therefore  propose 

A 

estimating  I(£)  by  the  observed  information  matrix  I  (£),  defined 

IS, 

as  the  negative  second  derivative  matrix  of  the  loglikelihood 
function  (2.9).  Justifications  for  this  approximation  are  given 
in  Efron  and  Hinkley  (1978).  Expressions  of  I(£)  suitable  for 
numerical  computation  are  summarized  below  under  computational  notes. 
Numerical  examples  of  this  approximation  are  shown  in  the  next 
two  sections. 

APPLICATION  TO  ARTHRITIS  KNOWLEDGE 

We  illustrate  our  method  using  responses  from  hospital 
patients  to  a  50  item  questionnaire  on  general  information  re¬ 
garding  arthritis.  The  questionnarie  was  completed  by  n  =  167  subjects 
Of  the  original  50  items,  3  were  deleted  since  their  parameters 
could  not  be  estimated,  and  we  thus  used  the  remaining  k  =  47  items. 


Estimates  of  (a.  ,8.)  and  there  covariances  are  summarized  in 
J  3 

Table  1. 

_ Insert_Table _ 1 _ about _h ere _ 

SIMULATION 

To  obtain  some  indication  of  performance,  our  method  was 
applied  to  a  randomly  generated  data  set,  Y,  using  n  =  200 
ability  parameters  randomly  selected  from  a  standard  normal 
distribution  and  ,k  =  50  pairs  of  item  parameters  (a^, 6j)  whose 
values  are  in  the  range  typically  encountered  in  pratice.  Figures 
1,  2  and  3  present  plots  of  the  estimated  versus  simulated  values 


of  a, 8,  and  0.  The  estimates  for  0  are  posterior  means  obtained 
from  (3.3)  with  £°  =  as  in  Rigdon  and  Tsutakawa  (1983).  We 
note  a  fairly  high  correlation  of  .988  and  .974  for  6  and  0, 
respectively,  but  a  somewhat  lower  one,  .881,  for  a.  The  estimated 
covariances  for  (a, 8)  are  summarized  in  Table  2  together  with 
standardized  errors  (a  -  a)/SD(a)  ani.  (8  -  B)/SD(B),  where  SD's 
are  square  roots  of  the  variances  approximated  by  I  1(c).  The 
simulation  was  repeated  using  n  =  400  and  k  =  25,  with  similar 
results  on  the  standardized  errors.  Although  these  results  appear 
quite  plausible,  until  we  obtain  more  experience  using  1(C)  the 
method  should  be  used  with  caution  for  data  sets  of  smaller  sizes 
or  when  the  model  may  not  be  appropriate. 

Insert  Figure  1,2,  &  3  and  Table  2  about  here 

COMPUTATIONAL  NOTES 

We  now  summarize  expressions  for  the  first  two  derivatives 


of  the  loglikelihood  function  (2.9)  and  the  posterior  expectation 
(3.1),  which  may  be  used  to  evaluate  I  (£)  and  to  maximize  T(C), 
respective ly . 


$ 


a 


'3 


d2UZ) 


30U36V 


I  {h  ,(i,u)  -  qty  (i,u)}  if  u  =  v 


l  {d_„(i,u,v)  -  g  (i,u)g.(i,v)>  if  u  f  v. 


for  u,v  =  1, . . . ,k. 


For  any  pair  (C #  5°) ,  define  the  posterior  expectations  of 


derivativies  and  their  products, 


gs(i 


»u)  =  gg(i,u,9)  p(9 ! yi»£° )  de, 


(7.9) 


hgt(i,u)  =  J  hgt(i,u,6)  p(0 |yi,£° )de. 


(7.10) 


dgt  (i,u)  =  j  ga(i,uf8)gt(i,u#e)p(0|yi,£°  )de  .  (7.11) 


These  integrals  are  indentical  to  (7.1)  -  (7.3)  when  £  =  , 


but  are  generally  not  identical  to  them  when  £  £  Z  .  The 


derivatives  of  T(£)  may  then  be  summarized  by 


3T(C)  n 

=  ,L  5i(i'ul 


(7.12) 


3TU) 


l  g,(i/u) 


(7.13) 


L3- 
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32T(£) 

3a  3a. 
U  V 


l  { h. .  (i  ,u)  -  d.. (i,u) }  if  u  =  v 


if  u  f  v. 


(7.14) 


32T(£) 

3a  3 8 
u  v 


l  {h.  _  (i ,  u)  -  d.-(i,u)>  if  u  =  v 


if  u  /  v, 


(7.15) 


32T(£) 

3  8  38 
U  V 


U  ,  V  “ 


I  (h22(i,u)  -  d22(i,u)  y  if  u  =  v 


if  u  f1  v, 


(7.16) 


We  note  that  the  numerical  solution  to  (3.2)  and  the 
evaluation  of  I  U)  are  quite  sensitive  to  the  accuracy  of  the 
numerical  approximation  of  integrals.  A  typical  integral  is 
a  posterior  expectation  and,  except  for  a  constant  factor, 
has  the  form 


k 

H(e)  p (e )  n  P(y.  .|  e  ,a  .,6  .)de, 
j=l  1:5  3  : 


(7.17) 


where  the  function  H  varies  from  integral  to  integral.  The 
missing  constant  factor  is  the  reciprocal  of  this  integral  when 
H(0)  =  1.  Since  p(0)  is  the  standard  normal  pdf,  the  integral 
has  the  Mth  Oder  Gauss-Hermite  approximation 
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M  _ 

I  H(/2  x.) 
1=1  *- 


k 

n  PfyiJ/?  x.,a  #b 
j=l  3 


j)w£ 


(7.18) 


where  (x. ,w. (x  ,w„)  ar®  the  nodes  and  weights, 
x  i.  in  m 

(See  Strout  and  Secrest  1966) .  Since  the  nodes  are  based  on 

the  prior  of  0.  and  not  its  posterior,  there  is  some  loss  in 

accuracy  when  the  posterior  is  displaced  from  the  prior.  To 

avoid  this  loss  one  can  select  the  nodes  around  the  posterior 

mean  nu  using  the  posterior  standard  deviation  c^  as  the  scale 

factor.  To  find  m.  and  c.  we  first  use  (7.18)  with  H(e)  =  e 

ii 

2  2 

for  nu  and  H(e)  ■  (e  -  nu)  for  c^.  Then  the  Gauss -Hermite 
approximation  to.  (7.17)  becomes 


✓2  c. 
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w, 
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x£) 


exp(x^) 


p(mi  +  /?  ci  x^) 


k  _ 

n  P ( y . . | m .  +  / 2  c.  xp , a . , 6 . ) . 
j  =  l  1  1  ^  J  ] 
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FIGURE  1 


ESTIMATED  VS.  ACTUAL  ALPHA 
FROM  SIMULATION  WITH 
N=200  AND  K=50 
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FIGURE  3 


ESTIMATED  AND  ACTUAL  THETAS 
FROM  SIMULATION  WITH 
N=200  AND  K=50 
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TABLE  I 

PARAMETER  ESTIMATES  AND  COVARIANCES 
FOR  ARTHRITIS  EXAMPLE 

q _ 6 _ Var  (q) _ Var(g,  | ) _ Var(B) 

1.  14  -2.44  0.1287  0.1910  0.349U 

1.96  -1.69  0.1837  0.0381  0.0310 

0.  62  -3.  96  0.0722  0.4022  2.5197 

1.84  -1.64  0.1552  0.0365  0.0353 

1.30  -2.03  0.1124  0.0988  0.1431 

1.41  -1.86  0.0981  0.0508  0.0754 

1.31  -1.85  0.0983  0.0700  0.1031 

2.  67  -1.28  0.3574  -0.0004  0.0111 

1.20  -1.85  0.0765  0.0579  0.1033 

2.12  -1.26  0.1855  0.0123  0.0192 

1.12  -1.84  0.0715  0.0625  0.1205 

0.73  -2.40  0.0436  0.1027  0.3838 

1.05  -1.77  0.0562  0.0457  0.1053 

0.  95  -1.86  0.0538  0.0592  0.1467 

0.65  -2.43  0.0418  0.1170  0.4909 

1.65  -1.21  0.0983  0.0064  0.0253 

0.70  -2.21  0.  0491  0.1162  0.4076 

0.  36  -3.89  0.0370  0.3704  4.1652 

0.69  -1.74  0.0431  0.0802  0.2588 

0.52  -2.18  0.  0400  0.1425  0.6913 

1.  52  -0.89  0.0728  -0.  0030  0.0220 

1.13  -0.98  0.0567  0.0151  0.0436 

1.  11  -0.  96  0.0536  0.0150  0.0443 

0.  67  -1.36  0.0432  0.0643  0.1909 

0.51  -1.45  0.0365  0.0871  0.3493 

0.68  -0.98  0.0374  0.0363  0.1159 

0.60  -0.91  0.0416  0.0523  0.1598 

0.  52  -1.05  0.0352  0.0593  0.2250 

1.10  -0.39  0.0563  0.0058  0.0338 

1.06  -0.40  0.0575  0.0085  0.0361 

0.91  -0.33  0.0462  0.0081  0.0444 

0.26  -1.06  0.0331  0.1350  0.9367 

0.  77  -0.17  0.0480  0.0074  0.0545 

0.70  -0.19  0.0452  0.0094  0.0648 

0.61  -0.18  0.0419  0.0121  0.0831 

0.67  0.04  0.0478  -0.0003  0.0664 

1.00  0.08  0.0649  -0.0034  0.0360 

1.29  0.  11  0.  0954  -0.0035  0.0258 

0.  68  0.44  0.  0506  -0.0242  0.0770 

1.01  0.38  0.  0758  -0.0176  0.0400 

0.46  1.  18  0.0466  -0.1052  0.3755 

0.42  1.33  0.0442  -0.1222  0.4979 

0.  30  2.17  0.  0444  -0.2974  2.3145 

0,75  1.28  0.0712  -0.0952  0.1897 

0.68  1.71  0.0734  -0.1503  0.3862 

0.54  2.48  0.0686  -0.2752  1.2395 

0.44  3.85  0.0840  -0.6664  5.5250 


TABLE  II 


COVARIANCES  AND  STANDARDIZED  ERRORS 
FROM  SIMULATION 


Item 

Var (a) 

A  A 

Cov(a,6) 

Var(e) 

Standardized  Error 

a  B 

1 

0.0913 

0.0303 

0.0320 

0.33 

0.52 

2 

0.0492 

0.0220 

0.0388 

-0.14 

1.78 

3 

0.1395 

0.0200 

0.0164 

0.75 

1.23 

4 

0.1042 

0.0369 

0.0347 

-0.48 

-0.96 

5 

0.3488 

0.0178 

0.0088 

1.59 

1.42 

6 

0.4080 

0.0176 

0.0082 

1.42 

1.29 

7 

0.1349 

0.0100 

0.0121 

0.91 

2.74 

8 

0.1032 

0.0115 

0.0151 

0.75 

2.33 

9 

0.1747 

0.0107 

0.0108 

0.77 

1.72 

10 

0.0581 

0.0153 

0.0268 

0.24 

1.19 

11 

0.0515 

0.0185 

0.0333 

-1.31 

0.77 

12 

0.0714 

0.0167 

0.0237 

-0.28 

0.48 

13 

0.1907 

0.0057 

0.0089 

0.49 

1.79 

14 

0.0746 

0.0055 

0.0160 

0.57 

2.32 

15 

0.1534 

-0.0029 

0.0086 

-0.35 

2.35 

16 

0.0496 

0.0092 

0.0255 

0.24 

-0.41 

17 

0.3044 

-0.0112 

0.0064 

1.87 

2.93 

18 

0.0561 

0.0051 

0.0202 

0.05 

-0.94 

19 

0.0855 

0.0215 

0.0166 

-0.70 

-0.78 

20 

0.0572 

-0.0009 

0.0178 

-0.07 

0.47 

21 

0.0589 

-0.0052 

0.0170 

-0.77 

0.28 

22 

0.0658 

-0.0064 

0.0155 

1.01 

0.75 

23 

0.0666 

-0.0076 

0.0155 

0.96 

0.94 

24 

0.0519 

-0.0057 

0.0195 

0.71 

0.57 

25 

0.1745 

-0.0192 

0.0101 

1.11 

1.12 

26 

0.0787 

-0.0138 

0.0150 

0.03 

0.49 

27 

0.0587 

-0.0085 

0.0176 

0.34 

-0.72 

28 

0.0905 

-0.0161 

0.0143 

0.93 

0.50 

29 

0.0390 

-0.0129 

0.0320 

-1.39 

0.70 

30 

0.0727 

-0.0225 

0.0195 

0.55 

0.92 

31 

0.0443 

-0.0288 

0.0440 

-1.35 

1.65 

32 

0.0495 

-0.0251 

0.0330 

-0.98 

0.55 

33 

0.1206 

-0.0341 

0.0197 

-0.01 

0.60 

34 

0.1315 

-0.0464 

0.0258 

-0.12 

0.81 

35 

0.0687 

-0.0346 

0.0316 

0.52 

0.11 

36 

0.0568 

-0.0216 

0.0261 

0.19 

-1.62 

37 

0.0781 

-0.0400 

0.0321 

0.03 

0.26 

38 

0.0716 

-0.0520 

0.0510 

-0.89 

1.47 

39 

0.0974 

-0.0392 

0.0261 

0.25 

-0.40 

40 

0.0647 

-0.0433 

0.0426 

0.11 

0.50 

41 

0.1214 

-0.0403 

0.0222 

0.69 

-1.04 

42 

0.0794 

-0.0460 

0.0370 

0.50 

0.35 

43 

0.1112 

-0.0482 

0.0287 

0.58 

-0.01 

44 

0.3021 

-0.0824 

0.0249 

0.95 

0.19 

45 

0.1651 

-0.0559 

0.0238 

1.23 

-0.85 

46 

0.0522 

-0.0511 

0.0533 

-0.42 

0.33 

47 

0.0909 

-0.0608 

0.0442 

1.25 

-0.57 

48 

0.0539 

-0.0566 

0.0714 

-0.63 

-0.31 

49 

0.1972 

-0.1049 

0.0475 

1.45 

-0.12 

50 

0.1587 

-0.1260 

0.0814 

0.24 

0.52 
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Lowry  AFB,  CO  80230 
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Department  of  Defense 

12  Defense  Technical  Information  Center 
Cameron  Station,  Bldg  5 
Alexandria,  VA  22314 
Attn:  TC 

1  Dr.  Craig  I.  Fields 
Advanced  Research  Projects  Agency 
1400  Wilson  Blvd. 

Arlington,  VA  22209 

1  Dr.  William  Graham 
Testing  Directorate 
MEPCOM/MEPCT-P 
Ft.  Sheridan,  IL  60037 

1  Jerry  Lehnus 
HQ  MEPCOM 
Attn:  MEPCT-P 
Fort  Sheridan,  IL  60037 

1  Military  Assistant  for  Training  and 
Personnel  Technology 

Office  of  the  Under  Secretary  of  Defens 
for  Research  &  Engineering 
Room  3D129.  The  Pentagon 
Washington,  DC  20301 

1  Dr.  Wayne  Sellman 

Office  of  the  Assistant  Secretary 
of  Defense  (MRA  &  L) 

2B269  The  Pentagon 
Washington,  DC  20301 

1  Major  Jack  Thorpe 
DARPA 

1400  Wilson  Blvd. 

Arlington,  VA  22209 

1  Dr.  Robert  A.  Wisher 
0USDRE  (ELS) 

The  Pentagon,  Room  3D 129 
Washington,  DC  20301 


Mis sour 1/Tsutakawa  (NR150-464)  16-Nov-83 

Civilian  Agencies 

1  Dr.  Susan  Chipman 

Learning  and  Development 
National  Institute  of  Education 
1200  19th  Street  NW 
Washington,  DC  20208 

1  Dr.  Arthur  Melmed 
724  Brown 

U.  S.  Dept,  of  Education 
Washington,  DC  20208 

1  Dr.  Andrew  R.  Molnar 
Office  of  Scientific  and  Engineering 
Personnel  and  Education 
National  Science  Foundation 
Washington.  DC  20550 

1  Dr.  Vern  W.  Urry 
Personnel  R&D  Center 
Office  of  Personnel  Management 
1900  E  Street  NW 
Washington,  DC  20415 

1  Mr.  Thomas  A.  Warm 
U.  S.  Coast  Guard  Institute 
P.  0.  Substation  18 
Oklahoma  City,  OK  73169 

1  Dr.  Joseph  L.  Young,  Director 
Memory  &  Cognitive  Processes 
National  Science  Foundation 
Washington,  DC  20550 
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Private  Sector 

1  Dr.  James  Algina 
University  of  Florida 
Gainesville,  FL  326 

1  Dr.  Erling  B.  Andersen 
Department  of  Statistics 
Studiestraede  6 
1455  Copenhagen 
DENMARK 

1  Dr.  Isaac  Bejar 
Educational  Testing  Service 
Princeton,  NJ  08450 

1  Dr.  Menucha  Birenbaum 
School  of  Education 
Tel  Aviv  University 
Tel  Aviv,  Ramat  Aviv  69978 
Israel 

1  Dr.  Werner  Birke 

Personals tammamt  der  Bundeswehr 
D-5000  Koeln  90 
WEST  GERMANY 

1  Dr.  R.  Darrell  Bock 
Department  of  Education 
University  of  Chicago 
Chicago,  IL  60637 

1  Dr.  Robert  Brennan 

American  College  Testing  Programs 

P.  0.  Box  168 

Iowa  City,  IA  52243 

1  Dr.  Glenn  Bryan 
6208  Poe  Road 
Bethesda,  MD  20817 

1  Dr.  Ernest  R.  Cadotte 
307  Stokely 

University  of  Tennessee 
Knoxville,  TN  37916 


1  Dr.  John  B.  Carroll 
409  Elliott  Rd. 

Chapel  Hill,  NC  27514 

1  Dr.  Norman  Cliff 
Dept,  of  Psychology 
Univ.  of  So.  California 
University  Park 
Los  Angeles,  CA  90007 
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Private  Sector 

1  Dr.  Allan  M.  Collins 

Bolt  Beranek  &  Newman,  Inc. 

50  Moulton  Street 
Cambridge,  MA  02138 

1  Dr.  Hans  Crombag 
Education  Research  Center 
University  of  Leyden 
Boerhaavelaan  2 
2334  EN  Leyden 
The  NETHERLANDS 

1  CTB/McGraw-Hill  Library 
2500  Garden  Road 
Monterey,  CA  93940 

1  Dr.  Dattpradad  Dlvgi 
Syracuse  University 
Department  of  Psychology 
Syracuse,  NE  33210 

1  Dr.  Fritz  Drasgow 
Department  of  Psychology 
University  of  Illinois 
603  E.  Daniel  St. 

Champaign,  IL  61820 

1  Dr.  Susan  Ember tson 
PSYCHOLOGY  DEPARTMENT 
UNIVERSITY  OF  KANSAS 
Lawrence,  KS  66045 

1  ERIC  Facility-Acquisitions 
4833  Rugby  Avenue 
Bethesda,  MD  20014 

1  Dr.  Benjamin  A.  Fairbank,  Jr. 
McFann-Gray  &  Associates,  Inc. 

5825  Callaghan 
Suite  225 

San  Antonio,  TX  78228 

1  Dr.  Leonard  Feldt 
Lindquist  Center  for  Measurment 
University  of  Iowa 
Iowa  City,  LA  52242 

1  Dr.  Richard  L.  Ferguson 
The  American  College  Testing  Program 
P.0.  Box  168 
Iowa  City,  IA  52240 


Private  Sector 

1  Univ.  Prof.  Dr.  Gerhard  Fischer 
Liebiggasse  5/3 
A  1010  Vienna 
AUSTRIA 

1  Professor  Donald  Fitzgerald 
University  of  New  England 
Armidale,  New  South  Wales  2351 
AUSTRALIA 

1  Dr.  Dexter  Fletcher 
University  of  Oregon 
Department  of  Computer  Science 
Eugene,  OR  97403 

1  Dr.  John  R.  Frederiksen 
Bolt  Beranek  &  Newman 
50  Moulton  Street 
Cambridge,  MA  02138 

1  Dr.  Janice  Gifford 
University  of  Massachusetts 
School  of  Education 
Amherst,  MA  01002 

1  Dr.  Robert  Glaser 

Learning  Research  &  Development  Center 
University  of  Pittsburgh 
3939  O’Hara  Street 
PITTSBURGH,  PA  15260 

1  Dr.. Bert  Green 
Johns  Hopkins  University 
Department  of  Psychology 
Charles  &  34th  Street 
Baltimore,  MD  21218 

1  Dr.  Ron  Hambleton 
School  of  Education 
University  of  Massachusetts 
Amherst,  MA  01002 

1  Dr.  Paul  Horst 
677  G  Street,  #184 
Chula  Vista,  CA  90010 

1  Dr.  Lloyd  Humphreys 
Department  of  Psychology 
University  of  Illinois 
603  East  Daniel  Street 
Champaign,  IL  61820 
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Private  Sector 

1  Or.  Earl  Hunt 
Dept .  of  Psychology 
University  of  Washington 
Seattle,  VA  98105 

1  Dr.  Jack  Hunter 
2122  Coolidge  St. 

Lansing.  MI  48906 

1  Or.  Huynh  Huynh 
College  of  Education 
University  of  South  Carolina 
Columbia,  SC  29208 

1  Dr.  Douglas  H.  Jones 
Advanced  Statistical  Technologies 
Corporation 
10  Trafalgar  Court 
Lawrencevllle,  NJ  08148 

1  Professor  John  A.  Keats 
Departaent  of  Psychology 
The  University  of  Newcastle 
N.S.W.  2308 
AUSTRALIA 
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Private  Sector 

1  Dr.  Charles  Lewis 
Faculteit  Sociale  Wetenschappen 
Rijksuniversiteit  Groningen 
Oude  Boteringestraat  23 
9712GC  Groningen 
Netherlands 

1  Dr.  Robert  Linn 

College  of  Education 
University  of  Illinois 
Urbana,  IL  61801 

1  Mr.  Phillip  Livingston 

Systems  and  Applied  Sciences  Corporatio 
6811  Kenilworth  Avenue 
Riverdale,  MD  20840 

1  Dr.  Robert  Lockman 

Center  for  Naval  Analysis 
200  North  Beauregard  St. 

Alexandria,  VA  22311 

1  Dr.  Frederic  M.  Lord 
Educational  Testing  Service 
Princeton,  NJ  08541 


1  Dr.  Wllllaa  Koch 

University  of  Texas-Austln 
Measurement  and  Evaluation  Center 
Austin,  TX  78703 

1  Dr.  Marcy  Lansman 

The  L.  L.  Thurstone  Psychometric 
Laboratory 

University  of  North  Carolina 
Davie  Hall  013A 
Chapel  Hill,  NC  27514 

1  Dr.  Alan  Lesgold 
Learning  R&D  Center 
University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  15260 

1  Dr.  Michael  Levine 
Department  of  Educational  Psychology 
210  Education  Bldg. 

University  of  Illinois 
Champaign,  IL  61801 


1  Dr.  James  Lumsden 

Department  of  Psychology 
University  of  Western  Australia 
Nedlands  W.A.  6009 
AUSTRALIA 

1  Dr.  Gary  Marco 
Stop  31-E 

Educational  Testing  Service 
Princeton,  NJ  08451 

1  Dr.  Scott  Maxwell 

Department  of  Psychology 
University  of  Notre  Dame 
Notre  Dame,  IN  46556 

1  Dr.  Samuel  T.  Mayo 

Loyola  University  of  Chicago 
820  North  Michigan  Avenue 
Chicago,  IL  60611 

1  Mr.  Robert  McKinley 
American  College  Testing  Programs 
P.0.  Box  168 
Iowa  City,  IA  52243 
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Private  Sector 

1  Dr,  Barbara  Means 
Huaan  Resources  Research  Organization 
300  North  Washington 
Alexandria,  VA  22314 

1  Dr,  Robert  Mislevy 
711  Illinois  Street 
Geneva,  IL  60134 

1  Dr.  Allen  Munro 

Behavioral  Technology  Laboratories 
1845  Elena  Ave,.  Fourth  Floor 
Redondo  Beach,  CA  90277 

1  Dr.  W.  Alan  Nlcewander 
University  of  Oklahoma 
Department  of  Psychology 
Oklahoma  City,  OK  73069 

1  Dr.  Donald  A  Norman 
Cognitive  Science,  C-015 
Univ.  of  California,  San  Diego 
La  Jolla,  CA  92093 

1  Dr.  Melvin  R.  Novlck 

356  Lindquist  Center  for  Measurment 
University  of  Iowa 
Iowa  City,  IA  52242 

1  Dr.  James  Olson 
WICAT,  Inc. 

1875  South  State  Street 
Orem,  UT  84057 

1  Wayne  M.  Patience 
American  Council  on  Education 
GED  Testing  Service,  Suite  20 
One  Dupont  Cirle,  NW 
Washington,  DC  20036 

1  Dr.  James  A.  Paulson 
Portland  State  University 
P.0.  Box  751 
Portland,  OR  97207 

1  Dr.  James  W.  Pellegrino 
University  of  California, 

Santa  Barbara 
Dept,  of  Psychology 
Santa  Barabara,  CA  93106 
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Private  Sector 

1  Dr.  Mark  D.  Reckase 
ACT 

P.  0.  Box  168 
Iowa  City,  IA  52243 

1  Dr.  Lauren  Resnick 
LRDC 

University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  1521 

1  Dr.  Thomas  Reynolds 
University  of  Texas-Dallas 
Marketing  Department 
P.  0.  Box  688 
Richardson,  TX  75080 

1  Dr.  Andrew  M.  Rose 
American  Institutes  for  Research 
1055  Thomas  Jefferson  St.  NW 
Washington,  DC  20007 

1  Dr.  Lawrence  Rudner 
403  Elm  Avenue 
Takoma  Park,  MD  20012 

1  Dr.  J.  Ryan 
Department  of  Education 
University  of  South  Carolina 
Columbia,  SC  29208 

1  PROF.  FUMIK0  SAMEJIMA 
DEPT.  OF  PSYCHOLOGY 
UNIVERSITY  OF  TENNESSEE 
KNOXVILLE.  TN  37916 

1  Dr.  Walter  Schneider 
Psychology  Department 
603  E.  Daniel 
Champaign,  IL  61820 

1  Lowell  Schoer 
Psychological  &  Quantitative 
Foundations 
College  of  Education 
University  of  Iowa 
Iowa  City.  IA  52242 

1  Dr .  Kazuo  Shigemasu 
7-9-24  Kugenuma-Kaigan 
Fujusawa  251 
JAPAN 
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Private  Sector 

1  Dr.  Edwin  Shir key 

Departaent  of  Psychology 
University  of  Central  Florida 
Orlando,  FL  32816 

1  Dr.  tfllliaa  Slas 
Center  for  Naval  Analysis 
200  North  Beauregard  Street 
Alexandria,  VA  22311 

1  Dr.  H.  Wallace  Slnaiko 
Program  Director 

Manpower  Research  and  Advisory  Services 
Saithsonian  Institution 
801  North  Pitt  Street 
Alexandria,  VA  22314 

1  Dr.  Kathryn  T.  Spoehr 
Psychology  Departaent 
Brown  University 
Providence,  RI  02912 

1  Dr.  Robert  Sternberg 
Dept,  of  Psychology 
Tale  University 
Box  11A,  Tale  Station 
New  Haven,  CT  06520 

1  Dr.  Peter  Stoloff 

Center  for  Naval  Analysis 
200  North  Beauregard  Street 
Alexandria,  VA  22311 

1  Dr.  Wllllaa  Stout 
University  of  Illinois 
Departaent  of  Mathematics 
Urbana,  IL  61801 

1  DR.  PATRICK  SUPPES 

INSTITUTE  FOR  MATHEMATICAL  STUDIES  IN 
THE  SOCIAL  SCIENCES 
STANFORD  UNIVERSITY 
STANFORD.  CA  94305 

1  Dr.  Harlharan  Swaainathan 
Laboratory  of  Psychoaetric  and 
Evaluation  Research 
School  of  Education 
University  of  Massachusetts 
Aaherst,  MA  01003 


Private  Sector 

1  Dr.  Kikuml  Tatsuoka 
Computer  Based  Education  Research  Lab 
252  Engineering  Research  Laboratory 
Urbana,  IL  61801 

* 

1  Dr.  Maurice  Tatsuoka 
220  Education  Bldg 
1310  S.  Sixth  St. 

Champaign.  IL  61820 

1  Dr.  David  Thissen 
Department  of  Psychology 
University  of  Kansas 
Lawrence,  KS  66044 

1  Dr.  Douglas  Towne 

Univ.  of  So.  California 
Behavioral  Technology  Labs 
1845  S.  Elena  Ave. 

Redondo  Beach,  CA  90277 

1  Dr.  Robert  Tsutakawa 
Department  of  Statistics 
University  of  Missouri 
Columbia,  MO  65201 

1  Dr.  J.  Uhlaner 
Uhlaner  Consultants 
4258  Bonavita  Drive 
Encino,  CA  91436 

1  Dr.  V.  R.  R.  Uppuluri 
Union  Carbide  Corporation 
Nuclear  Division 
P.  0.  Box  Y 
Oak  Ridge,  TN  37830 

1  Dr.  David  Vale 

Assessment  Systems  Corporation 
2233  University  Avenue 

Suite  310  j 

St.  Paul,  MN  55114  I 

1  Dr.  Kurt  Van  Lehn  j 

Xerox  PARC  j 

3333  Coyote  Hill  Road  j 

Palo  Alto,  CA  94304  \ 

I 

1  Dr.  Howard  Wainer 
Division  of  Psychological  Studies 
Educational  Testing  Service 
Princeton,  NJ  08540 

ii 
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Private  Sector 


l  Dr.  Michael  T.  Waller 
Departaent  of  Educational  Psychology 
University  of  Wisconsin — Milwaukee 
Milwaukee,  WI  53201 

1  Dr.  Brian  Waters 
HuaRRO 

300  North  Washington 
Alexandria,  VA  22314 

1  Dr.  Phyllis  Weaver 
2979  Alexis  Drive 
Palo  Alto,  CA  94304 


1  Dr.  David  J.  Weiss 
N660  Elliott  Hall 
University  of  Minnesota 
75  E.  River  Road 
Minneapolis,  MN  55455 

1  Dr.  Donald  0.  Weitzman 
Mitre  Corporation 
1820  Dolley  Madison  Blvd 
McLean,  VA  22102 

1  Dr.  Rand  R.  Wilcox 

University  of  Southern  California 
Departaent  of  Psychology 
Los  Angeles,  CA  90007 

1  Wolfgang  Wlldgrube 
Streltkraefteamt 
Box  20  50  03 
D-5300  Bonn  2 
WEST  GERMANY 


1  Dr.  Bruce  Williams 
Departaent  of  Educational  Psychology 
University  of  Illinois 
Urbana,  XL  61801 

1  Dr.  Wendy  Yen 
CTB/McGraw  Hill 
Del  Monte  Research  Park 
Monterey,  CA  93940 
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